Search for: All records

The meta-reinforcement learning (meta-RL) framework, which involves RL over multiple timescales, has been successful in training deep RL models that generalize to new environments. It has been hypothesized that the prefrontal cortex may mediate meta-RL in the brain, but the evidence is scarce. Here we show that the orbitofrontal cortex (OFC) mediates meta-RL. We trained mice and deep RL models on a probabilistic reversal learning task across sessions during which they improved their trial-by-trial RL policy through meta-learning. Ca²⁺/calmodulin-dependent protein kinase II-dependent synaptic plasticity in OFC was necessary for this meta-learning but not for the within-session trial-by-trial RL in experts. After meta-learning, OFC activity robustly encoded value signals, and OFC inactivation impaired the RL behaviors. Longitudinal tracking of OFC activity revealed that meta-learning gradually shapes population value coding to guide the ongoing behavioral policy. Our results indicate that two distinct RL algorithms with distinct neural mechanisms and timescales coexist in OFC to support adaptive decision-making.

 

Sea level rise (SLR) may impose substantial economic costs to coastal communities worldwide, but characterizing its global impact remains challenging because SLR costs depend heavily on natural characteristics and human investments at each location – including topography, the spatial distribution of assets, and local adaptation decisions. To date, several impact models have been developed to estimate the global costs of SLR. Yet, the limited availability of open-source and modular platforms that easily ingest up-to-date socioeconomic and physical data sources restricts the ability of existing systems to incorporate new insights transparently. In this paper, we present a modular, open-source platform designed to address this need, providing end-to-end transparency from global input data to a scalable least-cost optimization framework that estimates adaptation and net SLR costs for nearly 10 000 global coastline segments and administrative regions. Our approach accounts both for uncertainty in the magnitude of global mean sea level (g.m.s.l.) rise and spatial variability in local relative sea level rise. Using this platform, we evaluate costs across 230 possible socioeconomic and SLR trajectories in the 21st century. According to the latest Intergovernmental Panel on Climate Change Assessment Report (AR6), g.m.s.l. is likely to rise during the 21st century by 0.40–0.69 m if late-century warming reaches 2 ∘C and by 0.58–0.91 m with 4 ∘C of warming (Fox-Kemper et al., 2021). With no forward-looking adaptation, we estimate that annual costs of sea level rise associated with a 2 ∘C scenario will likely fall between USD 1.2 and 4.0 trillion (0.1 % and 1.2 % of GDP, respectively) by 2100, depending on socioeconomic and sea level rise trajectories. Cost-effective, proactive adaptation would provide substantial benefits, lowering these values to between USD 110 and USD 530 billion (0.02 and 0.06 %) under an optimal adaptation scenario. For the likely SLR trajectories associated with 4 ∘C warming, these costs range from USD 3.1 to 6.9 trillion (0.3 % and 2.0 %) with no forward-looking adaptation and USD 200 billion to USD 750 billion (0.04 % to 0.09 %) under optimal adaptation. The Intergovernmental Panel on Climate Change (IPCC) notes that deeply uncertain physical processes like marine ice cliff instability could drive substantially higher global sea level rise, potentially approaching 2.0 m by 2100 in very high emission scenarios. Accordingly, we also model the impacts of 1.5 and 2.0 m g.m.s.l. rises by 2100; the associated annual cost estimates range from USD 11.2 to 30.6 trillion (1.2 % and 7.6 %) under no forward-looking adaptation and USD 420 billion to 1.5 trillion (0.08 % to 0.20 %) under optimal adaptation. Our modeling platform used to generate these estimates is publicly available in an effort to spur research collaboration and support decision-making, with segment-level physical and socioeconomic input characteristics provided at https://doi.org/10.5281/zenodo.7693868 (Bolliger et al., 2023a) and model results at https://doi.org/10.5281/zenodo.7693869 (Bolliger et al., 2023b).